Generation and data management of a medical study using instruments in an integrated media and medical system

ABSTRACT

In general, a computer-implemented method is described for receiving one or more requests to generate a medical study, retrieving from one or more data repositories one or more medical study instruments, receiving a selection of a particular medical study instrument to use in the medical study, generating by one or more computers a list of one or more research collaborators invited to review the medical study, and generating by one or more computers a list of one of more participants invited to join the medical study.

CLAIM OF PRIORITY

This application is a continuation of and claims priority under 35U.S.C. § 120 to Ser. No. 14/874,260, filed Oct. 2, 2015, which is acontinuation of and claims priority under 35 U.S.C. § 120 to Ser. No.14/102,196, filed Dec. 10, 2013 (now U.S. Pat. No. 9,153,003), which isa continuation of and claims priority under 35 U.S.C. § 120 to U.S.patent application Ser. No. 13/867,284, filed Apr. 22, 2013 (now U.S.Pat. No. 8,627,221), which is a continuation of and claims priorityunder 35 U.S.C. § 120 to U.S. patent application Ser. No. 12/699,522,filed Feb. 3, 2010 (now U.S. Pat. No. 8,429,547), which claims priorityunder 35 U.S.C. § 119(e) to provisional U.S. Patent Application61/253,398, filed on Oct. 20, 2009, the entire contents of each of whichare hereby incorporated by reference.

BACKGROUND

Medical forms are used to collect data and information regarding apatient's symptoms and conditions. One technique for preparing a medicalform is to manually edit a pre-existing form (e.g., a form existing inMicrosoft Word™ format) with new or customized questions. The form isthen sent to review boards for review through a physical or electronicmailing. Additionally, once a form has been finalized, it may bepresented to a patient, study participant or other individual(collectively referred to as “patients” herein, without limitation, forpurposes of convenience). For example, physicians may present patientswith the forms when the patient visits the physician's office.Additionally, hardcopy (i.e., paper) versions of medical forms may bedistributed to patients for completion. For patient's who have notcompleted medical forms prior to the patient's examination, the patientmay often complete the medical form at the physician's office by fillingout a hardcopy of the form.

Frequently, the patient's responses to the questions included in themedical forms are entered into a computerized system by medicalpersonnel. In this case, in order for a physician to review thepatient's responses, the physician may access the computerized systemand view the answers to the questions, which is often a lengthy processof reviewing individual questions.

SUMMARY

The techniques described above fail to provide an integrated system forgenerating and validating an instrument (e.g., medical forms andquestionnaires) and collecting data associated with the validatedinstrument, such that the data is more readily accessible to researchersfor analysis.

In one exemplary embodiment, the techniques described herein provide,for example, an integrated system for physicians and other health careproviders to generate instruments that are reviewed and validatedthrough the use of threaded discussions and also to distribute thegenerated instruments to patients for completion. In another exemplaryembodiment, the results of completed instruments are stored in adatabase that is integrated with the system and that is accessible toresearchers and collaborators for review.

One aspect of the disclosure provides a computer-implemented method forreceiving one or more requests to generate a medical study, retrievingfrom one or more data repositories one or more medical studyinstruments, receiving a selection of a particular medical studyinstrument to use in the medical study, generating by one or morecomputers a list of one or more research collaborators invited to reviewthe medical study, and generating by one or more computers a list of oneof more participants invited to join the medical study.

Implementations of the disclosure may include one or more of thefollowing features. In some implementations, the method includesgenerating a graphical user interface that when rendered on a displaydevice renders a visual representation of a timeline, indicating, forone or more questions included in the selected medical study instrument,a question completion date. The method may also include generating byone or more computers a customized medical study instrument by:receiving one or more request messages indicating a type of question tobe included in the customized medical study instrument, question text tobe included in the customized medical study instrument and one or moreanswers to be included in the customized medical study instrument andassociated with the question text.

In some implementations, the method includes linking a score value toone or more questions included in the customized medical studyinstrument, and generating a scoring rule based on the linked scorevalues. The method may also include generating a notification messagingindicating that one or more questions included in the medical studyinstrument need to be answered, and sending the notification message toone or more participants of the medical study.

In other implementations, the method includes generating a graphicaluser interface that when rendered on a display device renders a visualrepresentation of a dashboard comprising one or more real-timeindicators of the medical study's progress, instrument scoring, patientenrollment, timelines and patient reminders. The method may also includegenerating by one or more computers a graphical user interface that whenrendered on a display device renders a visual representation of a forumfor review of the medical study instrument, the visual representation ofthe forum comprising: one or more text boxes for reviewers to entertext, the text boxes associated with one or more questions included inthe medical study instrument, and a link between the one or more textboxes and an associated question, selection of the link causing theassociated question to be rendered on the display device.

In yet other implementations, the method includes generating by one ormore computers a multimedia module to explain one or more features of aparticular question, and generating a link between the particularquestion and the multimedia module. The method may also includereceiving by one or more computers one or more responses to questionsincluded in the medical study instrument, retrieving by one or morecomputers one or more scoring rules, and executing by one or morecomputers the one or more scoring rules to determine a score of thereceived responses.

Another aspect of the disclosure provides a computer program productresiding on a computer readable storage medium, the computer programproduct comprising instructions for causing a computer to: receive oneor more requests to generate a medical study, retrieve from one or moredata repositories one or more medical study instruments, receive aselection of a particular medical study instrument to use in the medicalstudy, generate a list of one or more research collaborators invited toreview the medical study, and generate a list of one of moreparticipants invited to join the medical study.

Implementations of the disclosure may include one or more of thefollowing features. In some implementations, the computer programproduct includes instructions for causing the computer to generate agraphical user interface that when rendered on a display device rendersa visual representation of a timeline, indicating, for one or morequestions included in the selected medical study instrument, a questioncompletion date. The computer program product may also includeinstructions for causing the computer to generate a customized medicalstudy instrument by: receiving one or more request messages indicating atype of question to be included in the customized medical studyinstrument, question text to be included in the customized medical studyinstrument and one or more answers to be included in the customizedmedical study instrument and associated with the question text.

In other implementations, the computer program product includesinstructions for causing the computer to link a score value to one ormore questions included in the customized medical study instrument, andgenerate a scoring rule based on the linked score values. The computerprogram product may also include instructions for causing the computerto generate a notification messaging indicating that one or morequestions included in the medical study instrument need to be answered,and send the notification message to one or more participants of themedical study.

In yet other implementations, the computer program product includesinstructions for causing the computer to generate a graphical userinterface that when rendered on a display device renders a visualrepresentation of a dashboard comprising one or more real-timeindicators of the medical study's progress, instrument scoring, patientenrollment, timelines and patient reminders. The computer programproduct may also include instructions for causing the computer togenerate a graphical user interface that when rendered on a displaydevice renders a visual representation of a forum for review of themedical study instrument, the visual representation of the forumcomprising: one or more text boxes for reviewers to enter text, the textboxes associated with one or more questions included in the medicalstudy instrument, and a link between the one or more text boxes and anassociated question, selection of the link causing the associatedquestion to be rendered on the display device. The computer programproduct may also include instructions for causing the computer togenerate a multimedia module to explain one or more features of aparticular question; and generate a link between the particular questionand the multimedia module.

In still other implementations, the computer program product includesinstructions for causing the computer to receive one or more responsesto questions included in the medical study instrument, retrieve one ormore scoring rules, and execute the one or more scoring rules todetermine a score of the received responses.

Another aspect of the disclosure provides an apparatus comprising: aprocessor; and a computer program product residing on a computerreadable storage medium, the computer program product comprisinginstructions for causing the processor to: receive one or more requeststo generate a medical study; retrieve from one or more data repositoriesone or more medical study instruments; receive a selection of aparticular medical study instrument to use in the medical study;generate a list of one or more research collaborators invited to reviewthe medical study; and generate a list of one of more participantsinvited to join the medical study.

Implementations of the disclosure may include one or more of thefollowing features. In some implementations, the computer programproduct includes instructions for causing the computer to perform thefeatures described above, for example, instructions for causing thecomputer to generate a graphical user interface that when rendered on adisplay device renders a visual representation of a timeline,indicating, for one or more questions included in the selected medicalstudy instrument, a question completion date.

Another aspect of the disclosure provides a computer-implemented methodfor retrieving by one or more computers one or more answer options to anassociated question included in a medical study, with one or more of theanswer options linked to a color value indicative of a disability levelassociated with a medical issue, causing one or more computers todetermine for the one or more answer options a display color for theanswer option according to the color value, and generating by one ormore computers a graphical user interface that when rendered on adisplay device renders a first visual representation of the one or moreanswer options in the determined display color, the one or more answeroptions juxtaposed to a second visual representation of the associatedquestion.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a diagrammatic view of a system for generating and trackingmedical study instruments.

FIGS. 2, 3, 7, 15 are flowcharts of processes used by a medicalintegration system.

FIGS. 4-6, 8-14 and 16-18 are screen images of a graphical userinterface generated by the medical integration system.

DETAILED DESCRIPTION

Overview

The system described herein provides, for example, an integratedinformation and communication platform for outcome and evidence-basedmedical research. Using the system, researchers may design studies totest the efficacy of medical devices, medical instruments and medicaltreatments. Through the system, researchers may also collaborate withcolleagues to validate and to refine the study and invite patients toparticipate in the study. Patients may further access the study throughthe system.

In the exemplary embodiment described herein, through the use of portalsystems and dashboards, researchers have access to the progress of astudy and real-time feedback from the participants of the study. Thedescribed system collects data from the study and stores it in adatabase accessible to patients, physicians, nurses, health careproviders, experts, researchers, reviewers and other individuals(collectively referred to as “users” herein, without limitation, forpurposes of convenience) of the system, thereby enabling worldwidecollaborative research. As data is collected from the study, theexemplary system provides research analysis tools that enable users ofthe system to quantify outcome of a treatment under study by performing,for example, Rasch analysis, factor analysis, model testing, and avariety of statistical analysis to assess reliability, scaling andscoring of the results of the study. The exemplary system also includesdata mining tools to detect patterns in the collected data and convertcontinuous data into discrete categories of data.

FIG. 1 illustrates a particular exemplary embodiment described herein.Referring to FIG. 1, the illustrated network computer system 100includes a medical integration system (e.g., a server) 102 for makingconnections with users of the system 100, including patients 104, atclient systems 106, and health care professionals (e.g., physicians,medical clinic staff, research collaborators and research review boards)108, at client systems 110. In a preferred embodiment, the system mayoperate over a network 112, e.g., the Internet or other types ofnetworks. The medical integration system 102 operates as a servicerunning on, for example, a web server 114. In this embodiment, users104, 108 of the system 100 may connect to the medical integration system102 through a website or graphical user interface on the web server 114using client systems 106, 110.

Client systems 106, 110 include, for example, mobile devices (e.g.,cellular and mobile telephones, Blackberries™, iPhones™ and personaldigital assistants), personal computing devices (e.g., laptop anddesktop computers and iPads™), dedicated computing devices (e.g., kiosksystems installed in a doctor's office or medical clinic), andmedia-player-type devices. Client systems 110 also include furnitureequipped with a computing device. In a particular example, a chair in aphysician's office includes a mounted personal computing device (e.g., alaptop connected to the arm rest of a chair) through which users accessthe medical integration system 102 over the network 112. In thisexample, because numerous patients use the personal computing device, anultraviolet (“UV”) light may be included in the personal computingdevice and cast on the keyboard of the personal computing device toremove and destroy germs located on the personal computing device.

The illustrated medical integration system 102 includes an instrumentgeneration module 116, an instrument validation module 118, a datamining and research tools module 120, a procedure determination module122, and a patient flow module 124. Through a communication channel,including for example and without limitation a connection, a buffer or apath between various nodes in a network, the medical integration system102 may also access one or more databases 126 and one or more electronicmedical record (“EMR”) systems 128 to import data (e.g., medical datafrom external and/or internal data sources) into the medical integrationsystem 102. The components and modules 116, 118, 120, 122 and 124 of themedical integration system 102 and the web server 114 may be integratedor distributed in various combinations as is commonly known in the art.

Instrument Generation Module

In the exemplary embodiment described herein, users may generate studyinstruments (e.g., forms and questionnaires) for studies (e.g., medicalstudies) through the instrument generation module 116 of the medicalintegration system 102. In a preferred embodiment, an instrumentgeneration system includes an instrument generation module.

The instrument generation module 116 enables a user to define variousfeatures of a study, including a study description, a list ofinstruments used in the study, a study and/or instrument timeline,invitations for doctors to participate in the study, and a checklist ofcriteria for a patient to posses for inclusion in the study. Theinstruments may include pre-defined forms and questionnaires stored inthe medical integration system 102, and forms and questionnaires customdesigned for a particular study. The pre-defined instruments can beaugmented through the association of additional data, including labtests and observations provided by physicians, nurses, care givers, orfamily members.

FIG. 2 illustrates a particular exemplary embodiment described herein.Referring to FIG. 2, the medical integration system 102 executes 130 andcombines various processes including the following. In this exemplaryembodiment, the medical integration system 102 generates 132 aninstrument, as described in further detail below. The medicalintegration system 102 also receives 133 patients' responses to competedinstruments and stores 134 the patients' responses in the database 126.A scoring system including scoring rules defines how a patient'sresponses to answers in an instrument are valued or “scored,” asdescribed in further detail below. The medical integration system 102retrieves 135 from the database 126 one or more scoring rules andgenerates 136 an instrument score by applying the scoring rules to theanswers associated with an instrument.

In this embodiment, the medical integration system 102 performs dataanalysis on the stored patient responses and generates visualrepresentations of the data analysis which is presented to users of themedical integration system 102 through a graphical user interface. Themedical integration system 102 receives 137 a request from a user fordata matching criteria that the user has entered. The medicalintegration system 102 generates, for example, a query based on theentered criteria and queries 138 the database 126 for data matching theentered criteria included in the query, as described in further detailbelow. The medical integration system 102 also generates 139 a visualrepresentation of the data matching the entered criteria.

FIG. 3 illustrates a particular exemplary embodiment described herein.Referring to FIG. 3, the medical integration system 102 generates 140 astudy instrument, as follows. The medical integration system 102receives 142 a description of the study entered into the system 102through a graphical user interface 144, an example of which isillustrated in FIG. 4. The description of the study may include studydetails (e.g., information about a study), the name of the study, studyduration (e.g., a length in time which the study is available for apatient to complete), a number of patients to be included in the study,and clinic accessibility information (e.g., whether physicians within aclinic are granted permissions to view a study).

In this embodiment, the medical integration system 102 receives 148various criteria for a patient to possess in order to be included in thestudy. The system 102 may generate a graphical user interface 150, anexample of which is illustrated in FIG. 5. Through the graphical userinterface 150, the user may enter various patient criteria, includingdiagnosis codes, International Statistical Classification of Diseasesand Related Health Problems (“ICD”) codes (e.g., ICD-9 and ICD-10codes), Current Procedural Terminology (“CPT”) codes and other variousdiagnosis information and treatment information and various inclusive orexclusive criteria (e.g., age information, height information, weightinformation and smoker status information). The system 102 includes, forexample, a utility to search an ICD code library based on an ICD codedescription or a partial ICD code to find the correct code.Additionally, the system 102 may generate a list of predefined criteriaand displays this generated list for the user during generation of theinstrument.

In the exemplary embodiment described herein and referring back to FIG.3, the medical integration system 102 receives 154 a selection of one ormore instruments to be used in the study. The instruments includepre-defined forms, which may be standard for a particular industry, forexample the medical industry. The instruments may also be customgenerated, as described in further detail below. The medical integrationsystem 102 generates 158 an instrument timeline, which specifies thetime (e.g., a number of days, weeks or months) in which a particularquestion in the instrument should be answered by a patient participatingin the study. The timeline may also include the amount of time aninstrument is made available to the user to complete.

FIG. 6 illustrates a particular exemplary embodiment described herein.Referring to FIG. 6, the medical integration system 102 generates agraphical user interface 160 through which the timeline 162 may bedisplayed. In this embodiment, the illustrated timeline 162 includes ahorizontal axis 164 specifying a number of months relative to the startdate of the study for which a particular question should be completed bya patient. The illustrated timeline 162 may also include a vertical link166, which is capable of moving along the horizontal axis 164 to selecta particular month (e.g., two months or four months) in which aparticular question should be completed by a patient. As user herein andthroughout the application, the term “link” includes, withoutlimitation, pointers, Uniform Resource Location links, a reference orpointer to physical data, and a file that serves as a reference toanother file.

In the exemplary embodiment described herein and referring back to FIG.2, the medical integration system 102 generates 170 invitation messagesinviting physicians to join a study. The system 102 may include aninvitation module, through which users invite physicians and otherhealth care providers (collectively referred to as “physicians” herein,without limitation, for purposes of convenience) to participate in thestudy and to enroll their respective patients in the study. Theinvitation module generates, for example, a list of physiciansassociated with the user's hospital, clinic and/or research facility.The invitation module may also generate a list of physicians who havepreviously participated in studies generated by the medical integrationsystem 102.

Through the invitation module, the user may select from the generatedlists physicians to invite to join the study. To find physicians toinvite to join the study, the user may also search the system 102 forthe names of individual physicians to participate in and to collaboratein the study. In addition to inviting individual physicians, the usermay define groups of physicians to participate in the study ascollaborators, observers, and students of the instrument. In thisembodiment, the medical integration system 102 stores the names ofgroups of physicians that have previously been defined and invited tojoin other studies. The medical integration system 102 allows the userto access and search these pre-defined groups of physicians and invitegroups of physicians to join the user's study. Through the invitationmodule, a user may also invite a physician not currently enrolled in thesystem 102 to participate in the study. In this preferred embodiment,the invitation module sends an invitation message (e.g., an email orother electronic message) to join the study to the invited physicians.

In this exemplary embodiment, the medical integration system 102generates 174 a study summary, including the information (e.g., studydescription information, patient criteria information, selectedquestionnaire information, timeline information, and invited physicianinformation) input into the medical integration system 102 by the user.The system 102 may receive from the user a verification messageindicating that the summary information is accurate. The medicalintegration system 102 generates 178 the instrument and sends theinvited physicians a copy of the instrument and a list of the userspecified patient criteria. The medical integration system 102 may alsorecord the completion date by which specific questions in the instrumentshould be answered according to the timeline 162. The medicalintegration system 102 may also send alert messages (e.g., electronicmessages, text messages or short message service (“SMS”) messages) ofthe upcoming completion dates to the participating patients.

In an exemplary embodiment, the medical integration system 102 alsoallows the user to specify whether “skipped questions” are allowed forthe instrument and/or the number of questions to be completed before aninstrument score is calculated. In a particular example, the user mayspecify whether the instrument allows a percentage of questions to beskipped or whether all questions in the instrument need to be completed.

Frequently, Internal Review Boards (“IRBs”) require a patient to fillout a consent form prior to participating in a study or completing aninstrument. For this case, the medical integration system 102 includes aconsent form module through which users upload one or multiple consentforms to be presented to the patient before participating in the studyand/or before starting an instrument.

In an exemplary embodiment, once a generated instrument has beenfinalized, the medical integration system 102 provides the user with theoptions to start the study or to leave the study open for editing andtrial runs. When the study is left open for trial runs and editing, forexample, the medical integration system 102 may generate an online forumfor collaboration between the study generator and other participatingusers. Additionally, in this example when the study is left open fortrial runs and editing, the study instrument is editable, meaning thatthe criteria and the study options described above are editable. If theuser selects the option to start the study, the system 102 may send theselected patients, participants and collaborators an invitation messageto view and participate in the study by completing the instrument.

In another exemplary embodiment, a study may include a customizedinstrument generated through the medical integration system 102. Usersare provided a secure space to generate medically-based questionsoriginating from various question styles used by standard instruments.Multimedia modules may be attached to the questions to make thequestions more understandable for the patients completing theinstrument.

FIG. 7 illustrates a particular exemplary embodiment described herein.Referring to FIG. 7, the medical integration system 102 generates 190 acustomized instrument, as follows. In this illustrated example, themedical integration system 102 receives 192 one or more questions andassociated answers from a user. The medical integration system 102 alsoreceives 194 from the user a scoring system associated with thequestions and answers. The scoring system assigns a score value to eachof the answers associated with a question, enabling a physician orresearcher to assess the study results. The medical integration system102 generates 196 scoring rules based on the numerical scores associatedwith answers. In some examples, the scoring rules add the numericalscores associated with completed answers. In this illustrated example,the scoring rules may calculate a weighted average for the numericalscores associated with the completed answers by assigning the questionsa weighted value indicating the importance of the question relative toother questions. The medical integration system 102 prompts the usergenerating an instrument with a series of questions to determine thedesired type of score system and may generate a cautionary notificationif the user later applies an analytical procedure inappropriate for theselected score system (e.g., average age of study participants may bevalid and important to a study, but average social security number isneither valid nor important to the study).

In generating a customized instrument, a user may select a type ofquestion from a list of question types generated by the system 102. Thetypes of questions include, for example, multiple-choice questions(e.g., one question with multiple number responses), a “Yes/No” question(e.g., one question with “Yes” or “No” as responses), a “True/False”question (e.g., one question with “True” or “False” as responses), ascaling question (e.g., one question and a sliding scale bar throughwhich a user selects an answer located between two values, N1 and N2),an avatar/diagram question (e.g., an interactive question in which theuser selects various sections of an image or freely draws on varioussections of the image), a free response question in which a user inputsa predetermined amount of characters into a multi-line textbox (e.g.,“describe in 1000 characters or less your symptoms”).

FIG. 8 illustrates a particular exemplary embodiment described herein.Referring to FIG. 8, the medical integration system 102 generates agraphical user interface 210 for a user to define custom questions foran instrument. In section 212 of the illustrated graphical userinterface 210, the user may select the type of question to be added tothe instrument. In this particular embodiment, the user enters questiontext (e.g., “do you have difficulty opening a tight jar?”) in section214 of the illustrated graphical user interface 210. In section 216 ofthe illustrated graphical user interface 210, the user is provided theoption of adding a multimedia module (e.g., an image, an audio file oran image) to the question to assist a patient in understanding orinterpreting the question. In this particular embodiment, the system 102receives from the user's client system 106 request messages includingdata indicative of custom questions to be included in the instrument.

FIG. 9 illustrates a particular exemplary embodiment described herein.Referring to FIG. 9, a multimedia module 220 may be linked to aquestion. In this particular embodiment, the illustrated multimediamodule 220 includes a visual image 222 and an accompanying audio file224.

Referring back to the illustrated example of FIG. 8, section 226 of theillustrated graphical user interface 210 allows the user to associatethe question with a numerical score (i.e., numbers) and a score system,including, for example, a nominal system 228, an ordinal system 230, aninterval system 232 and a ratio system 234. In a nominal score system,numerical identifiers are associated with a response (e.g., 1=blue,2=red or 1=red, 2=blue). In an ordinal score system, the numericalscores are ordered in a meaningful way (e.g., 1=no pain, 2=some pain,3=severe pain). In an interval score system, the difference betweennumerical scores is significant (e.g., a 20 degree temperaturedifference between answers relating to temperate correlates with a fixedamount of heat). In a ratio score system, the numerical scores representa quotient, with the quotient indicative of the relative importance ofan answer.

Numerical scores may be associated with question responses to facilitatescoring of an instrument and mathematical and/or statistical analysis.Frequently, studies are designed to test the efficacy of a medicalprocedure, treatment or device. Accordingly, in a preferred embodiment,numerical scores are associated with questions, where a higher numericalscore indicates that the treatment is effective and a lower numericalscore indicates that the treatment is not effective or is lesseffective. In this embodiment, an instrument includes the followingquestion: “Did you experience an improvement in your symptoms followingthe treatment?” Answers associated with this question include, forexample, a “Yes” answer, a “Moderately” answer, and a “No” answer. Anumerical score of “3” is associated with the “Yes” answer. A numericalscore of “2” is associated with the “Moderately” answer. A numericalscore of “1” is associated with the “No” answer. In the illustratedexample of FIG. 8, the answers 236, 238, 240, 242, 244 associated with aquestion are assigned a numerical score 246, 248, 250, 252, 254 based onthe selected score system 228, 230, 232, 234. Additionally, in thisexample, a text box 256, 258, 260, 262 and 264 is provided in which theuser enters text corresponding to the answers associated with thequestion.

In the exemplary embodiment described herein, the answers to questionsmay be associated with a color value indicative of a feature of theanswer (e.g., answers directed toward a level of pain experienced by apatient or a level of difficulty the patient experiences in performingan activity). When the medical integration system 102 generates visualrepresentations of the answers to a question, the visual representationsmay be color coded based on the color value associated with an answer.

FIG. 10 illustrates a particular exemplary embodiment described herein.Referring to FIG. 10, the medical integration system 102 generates agraphical user interface 270 including, for example, answers to multiplequestions. In this embodiment, the “no difficulty” column 272 of answersis color coded green. The “mild difficulty” column 274 of answers iscolor coded yellow. The “moderate difficulty” column 276 of answers iscolor coded orange. The “severe difficulty” column 278 of answers iscolor coded light red. The “unable to complete” column 280 of answers iscolor coded bright red.

A color value system may be executed by the medical integration system102 in generating the illustrated graphical user interface 270 with thevarious colored columns of answers 272, 274, 276, 278, 280 according tothe following color value rules, for example. Answers with a color valueof “1” are assigned a color code of green, indicating that the physicalaction (e.g., preparing a meal) specified by the question is easy forthe patient to perform. Answers with a color value of “2” are assigned acolor code of yellow, indicating that the physical action specified bythe question is mildly difficult for the patient to perform. Answerswith a color value of “3” are assigned a color code of orange,indicating that the physical action specified by the question ismoderately difficult for the patient to perform. Answers with a colorvalue of “4” are assigned a color code of light red, indicating that thephysical action specified by the question is severely difficult for thepatient to perform. Answers with a color value of “5” are assigned acolor code of bright red, indicating that the patient is not able toperform the physical action specified by the question.

In the exemplary embodiment described herein, an instrument includesvarious sections. A section includes, for example, its own set ofinstructions that pertain to a grouping of questions. A section alsoincludes one or more associated questions. The types of instrumentsinclude patient instruments and physician instruments. Patientinstruments are designed to be filled out by or on the behalf of apatient. Physician instruments are designed to be filled out by or onbehalf of a physician in regards to a patient. Instruments may betranslatable to the primary language of the user and are translatable inreal time, based on a selected language of the user. Through aninstrument translation module, the medical integration system 102enables the generation of multi-lingual studies.

In one exemplary embodiment, the questions included in an instrument areassociated with alerting features. The medical integration system 102generates alerts by enabling physicians to establish answer thresholdvalues for a question or an instrument. In this embodiment, if a patientfills out an instrument and the patient's answer to a particularquestion or questions exceeds a threshold value, the medical integrationsystem 102 generates and sends the physician a notification message(e.g., an email or an alert icon is posted on a physician dashboardgraphical user interface). In a particular example, for the question “ona scale of 1-10 how much pain do you experience in your knee when youwake up in the morning?,” a physician establishes an answer thresholdvalue of 8, indicating that system 102 generates and sends anotification message to the physician when a patient selects a value of8, 9 or 10 in answering this question. The notification messages may beposted on a dashboard graphical user interface generated by the medicalintegration system 102, providing physicians with a quick view ofresponses to a physician's instrument without the physician having toreview many instruments to ascertain where patient responses exceedpredefined answer threshold values.

In another exemplary embodiment, instruments may be assigned apredetermined availability date: the date on which the instrumentbecomes available to the user. In one particular example, this date isset directly through a value being entered into a textbox by a user. Inanother particular example, the date is indirectly set by a reference toa number of days, weeks, months or years (e.g., 10 weeks from today).Instruments may also be assigned a duration of availability. During thisduration (e.g., a number of days, weeks, months or years), theinstrument is available for a user to complete.

Instrument Validation Module

In the exemplary embodiment described herein, when a customizedinstrument is generated, a new set of questions included in thecustomized instrument are reviewed, rated and validated in many reviewphases, with an individual phase modifying and/or eliminating questionsuntil the instrument is final. Frequently, associations, institutionsand clinics assemble expert panels to validate the custom generatedinstruments prior to the instruments' deployment to patients. In thiscase, the medical integration system 102 may generate online reviewforums to serve a global set of researchers or a singleorganization/clinic or physicians, reviewers and experts (collectivelyreferred to as “panel members” herein, without limitation, for purposesof convenience). The review forums allow a physician or researcher whogenerated the study (collectively referred to as “submitter” herein,without limitation, for purposes of convenience) to submit the questionsto be refined to the panel members in a secure, distributed and onlineenvironment. Through the review forums, for example, questions areviewed, commented on, rated and verified collaboratively by the panelmembers.

FIG. 11A illustrates a particular exemplary embodiment described herein.Referring to FIG. 11A, the medical integration system 102 generates agraphical user interface 300 that includes, for example, a visualrepresentation of a virtual forum for the panel members to collaborateand review an instrument. In this illustrated example, section 302 ofthe graphical user interface 300 includes a list of postings 304, 306,308, 310 made by the various panel members for an instrument. Theillustrated postings 304, 306, 308, 310 include threaded discussionslinked to a particular question or feature of the instrument (e.g., theinstrument timeline). Panel members collaborate with one another forprivate discussions using threaded discussion boards before passinginformation onto the submitters of the instrument. As the submittersrespond to the postings 304, 306, 308, 310, the submitters' responsesare added to the threaded discussion.

In one exemplary embodiment, the review forums include a rating system,through which panel members access tools to rate an instrument,including a list of predefined questions to consider in reviewing theinstrument. The rating system enables the panel members to rank thequality of a question and the question's associated score system andnumerical score.

In another exemplary embodiment, the review forums also include ascoring validation system to validate the scoring system associated withan instrument. Through the scoring validation system, panel members mayreview and comment on the validity of the scoring system associated withthe instrument and the validity of the numerical scores assigned toindividual questions within an instrument.

When a submitter and panel members have completed review of aninstrument (e.g., single or multi-centered studies), the medicalintegration system 102 permits the electronic generation of trialstudies to further validate the instrument and its questions and scoresystem. During trial studies, for example, the medical integrationsystem 102 engages patients to participate in the trial study, byemailing the finalized instrument to patients and soliciting feedback(e.g., feedback regarding study length or understandability of aquestion). Patients may provide feedback through the review forums,which include a section for a clinical trial threaded discussion.

In the exemplary embodiment described herein, through the medicalintegration system 102, a finalized instrument (i.e., a finalizedcustomized and non-customized instrument) may be sent to patients toparticipate in the study. Patients are selected to participate in astudy by the patient's physician, based on the patient's qualifications,including, for example, age and gender. Physicians are able to enterpatient name and other identifying information (e.g., email address andhome address) into the medical integration system 102. In thisembodiment, the medical integration system 102 also queries datarepositories, including EMR systems, for the names of patients qualifiedto participate in a study. The medical integration system 102 maygenerate a list of the patients qualified to participate in a study andmay enable a user to link (e.g., through a pointer) a selected patientto a particular study. As a result of the link, the system 102 maygenerate a notification message notifying the patient that he has beenselected to participate in a study. In some examples, the patient isnotified by an electronic message sent to the patient's electronic mailaddress. In other examples where the patient has a user account with themedical integration system 102, the patient is notified through adashboard message or alert displayed in the patient's dashboardgraphical user interface when the patient logs into the medicalintegration system 102.

Referring to the illustrated example of FIG. 10, the medical integrationsystem 102 may generate a dynamic instrument form graphical userinterface 270 through which users view and complete an instrument. Theillustrated dynamic instrument form graphical user interface 270 mayinclude a form progress indicator 282, which displays a portion of theinstrument completed by the user. In this illustrated example, the formprogress indicator 282 also displays a visual representation of apercentage of questions completed.

The illustrated dynamic instrument form graphical user interface 270also includes, for example, an auto-scroll feature (not shown), whichallows users to easily navigate an instrument without the use of ascrollbar or a mouse wheel. Upon completion of a question within aninstrument, the page automatically scrolls to the next unansweredquestion.

The dynamic instrument form graphical user interface 270 includes, forexample, a read-only mode for the review of past instrument results. Thedynamic instrument form graphical user interface 270 also includes atry-it mode which allows a user to view, fill out and score aninstrument without any data being recorded in the database 126. Try-itmode allows physicians who are unfamiliar with an instrument to fullyunderstand the instrument before assigning the instrument to patients.

In one exemplary embodiment, a skipped question manager may monitorskipped questions within an instrument. When a user skips a question,the manager marks and notifies the user (e.g., by a prompt box) beforefinal submission of the instrument to the system 102. In anotherexemplary embodiment, an audio reader feature within the medicalintegration system 102 returns and plays an audio rendition of a currentquestion as an instrument is being completed. This feature can betoggled on or off by the user. The audio files may be recorded andstored on the medical integration system 102 in a digital format andrelate to the respective question. Multi-language support is providedfor the audio recordings.

Frequently, multiple instruments may include questions that containsimilar content and/or the same content. In this case, a condensesimilar question module condenses into a single view or graphical userinterface questions across a single instrument or across multipleinstruments that are similar in type and responses. The condensed viewis then presented to the user. Through condensed views, the page size ofan instrument may be reduced, generating the appearance of a smallerinstrument that is more manageable for a user to answer. In a preferredembodiment, the condense similar question module executes rules thatmatch pre-defined key words to words included in a question by comparingone or more characters included in the question to the pre-definedkeywords. If multiple questions include characters matching (e.g., exactmatch, similar match or some variation thereof) the predefined keywordsincluded the rule, the module generates a condensed view of the multiplequestions including the matching characters.

FIG. 11B illustrates a particular exemplary embodiment described herein.Referring to FIG. 11B, the system 102 generates a graphical userinterface 312 that when rendered on a display device renders visualrepresentations of eight questions 311 a-311 h as a condensed question313 in a “carousel” form. The graphical user interface 312 includes, forexample, links 314 a, 314 b. In this illustrated example, when a userselects a link 314 a, 314 b, the system 102 generates another graphicaluser interface which displays the next question causing the view of thequestions 311 a-311 h to “rotate” the next question.

In one exemplary embodiment, the condense similar question module maygenerate follow up or adaptive questions, which are presented after auser has responded to a question. These adaptive questions ensure thatthe response a patient provides for a question corresponds to thevarious questions across multiple instruments. Through the condensesimilar question module, for example, a user provides answers forquestions across various instruments. Through these provided answers,the system 102 generates scores for questions across the variousinstruments, based on the questions' specified scoring system and theuser's provided answers.

In another exemplary embodiment, a reward point module allows users toearn points for the completion of an instrument. As a user completes aninstrument, a predetermined number of points are issued and associatedwith the user's account. At the end of a study, the accumulated numberof points may be applied towards the purchase of items. In a particularexample, as a physician generates a study instrument, the physician mayassign a reward package to the study. A reward package maps a number ofpoints to one or more questions included in the instrument.

Control Panel

In the exemplary embodiment described herein, users may access themedical integration system 102 through a control panel. Once a user haslogged into the medical integration system 102 through the controlpanel, users are presented with information to conduct studies, organizepatients and manage data.

FIG. 12 illustrates a particular exemplary embodiment described herein.Referring to FIG. 12, the medical integration system 102 generates agraphical user interface 320 including, for example, a control panel 321or a dashboard graphical user interface. In the illustrated example, asection 322 of the control panel 321 notifies a user of a number ofpatients enrolled in the medical integration system 102. Another section324 of the control panel 321 notifies a user of the number of studies inwhich the user is a participant. In this illustrated example, anothersection 326 of the control panel 321 notifies the user of the number ofunfinished instruments the user needs to complete. Another section 328of the control panel 321 notifies the user of a number of pendingstudies the user needs to join.

In one exemplary embodiment, an account information section of thecontrol panel allows users to view and edit user account information,including the following information: first and last name information,gender information, race information, primary language information, dateof birth information, and contact information (i.e., street address,city, state, zip code, email address, telephone number, mobile numberand fax number). In another exemplary embodiment, through the controlpanel, a user changes the user's account password by inputting theoriginal password, a new password and confirming the new password.Through the control panel, the user may specify the frequency (e.g.,once, once a day or once every twelve hours) for which notificationmessages (e.g., emails) regarding the tardiness of an instrument'scompletion are sent.

Another section of the control panels allows, for example, a user toview the available instruments included in the database. Through aselectable link, a user may choose to view instruments, includingspecialty instruments (e.g., forms that are specific to knee issues) andcustom instruments (e.g., instruments that have been customized for aclinic or instruments that are custom generated).

The control panel may also enable a user to manage studies by viewinginformation about studies, participating in studies and viewing studyinvitations, for example. The control panel may provide a search boxthrough which users of the medical integration system 102 search forvarious studies and instruments.

FIG. 13 illustrates a particular exemplary embodiment described herein.Referring to FIG. 13, the medical integration system 102 generates agraphical user interface 340 including a manage studies section 341 ofthe control panel. In this illustrated example, the user is providedwith a list of the studies generated by the user and the studies inwhich the user is a participant. The list includes, for example, thestudy name 344, a link 346, selection of which displays various detailsassociated with the study, the number of patients in the study comparedto a goal number of patients needed in the study 348, complianceinformation 350 (e.g., a percentage of instruments that have beencompleted), a study start date 352, a study end date 354, IRB statusinformation (e.g., a stage of the IRB approval status), whether thestudy has been sent to the IRB, whether the study is pending IRBapproval and communication, whether the IRB has approved the study, andwhether the study is available to patients. The pending instrumentsportion of the control panel includes, for example, a listing all thepending instruments the user must complete.

The control panel may also provide the user with a list of pendinginstruments, including for example the following information: the nameof the instrument, the patient to which the instrument has beenassigned, the date the instrument became available, the date theinstrument access expires, a numerical representation of the number ofdays left before the instrument access expires, a current status of theinstrument (e.g., the instrument has not been started, the instrumenthas been started but not completed), a list of the questions in theinstrument that have been completed and have a selected response, and alink to the instrument to begin completion.

In the exemplary embodiment described herein, the control panel mayprovide the user with a “study invitations” list: a list of the studiesthat the user has been invited as a participant. The study invitationslist includes, for example, the study name and a link to view variousdetails associated with the study, the name of the physician whogenerated the study, a brief description of the study, and a link 356through which the user accepts or declines the invitation to participatein the study. The control panel may also include a section for users toassign instruments to patients and doctors. By assigning an instrumentto a patient or a doctor, the patient or doctor is notified of theassigned instrument and of the need to complete the instrument.

In one exemplary embodiment, an instrument history section of thecontrol panel displays a visual representation of the instrumentscompleted by a patient and various completed instrument information,including, for example, the name of the instrument, the name of thepatient who completed the instrument, the name of the physician whoassigned the instrument, the date the instrument closed accessibility,the current status of the instrument, and a view of the individualresponses of the instrument.

In another exemplary embodiment, the control panel may also include aninstrument profile section which displays a visual representation of apre-defined collection of instruments that are grouped together as aprofile. These collections of instruments are used for ease of assigningcommonly occurring instruments to patients. In one particular example, acollection of instruments, such as the SF-12 instrument, the KOOSinstrument and the Marx instrument, used for new patients are collectedinto a profile called “New Patient.”

The control panel may also include a manage patients section foradding/editing/deleting a patient-physician relationship. The managepatients section includes, for example, a search box through which theuser searches for patients based on first or last name, the clinic withwhich the patient is associated, and the gender of the patient. Once thesystem 102 locates a patient, a user may edit the information associatedwith the patient and/or delete the patient information and instrumenthistory associated with the patient.

In the exemplary embodiment described herein, an overdue instrumentssection of the control panel displays a list of the patients that haveat least one overdue instrument (e.g., an instrument that has a statusof “not started” or “not completed”) and associated information,including for example the last name of the patient, the first name ofthe patient, the name of the clinic with which the patient isassociated, an overdue count (i.e., the number of instruments that aredeemed overdue), and a link to view the individual instruments that areoverdue. Through selection of the link associated with an overdueinstrument, the user has the option of extending the duration of time inwhich a patient has to complete the instrument or deleting theinstrument from the list of instruments assigned to the patient.

The similar questions section of the control panel displays, forexample, similar questions from other instruments based on apredetermined selected question included in a particular instrument asfollows. The control panel displays a list of instruments available foruse in a study. Based on the user's selection of a particularinstrument, the medical integration system 102 may display the questionsincluded in the selected instrument. Based on the user's selection of aparticular question within the selected instrument, the medicalintegration system 102 may also determine other instruments with similarquestions and display associated information, include for example: thename of the instrument including the similar questions, the questionnumber within the respective instrument, the text of the actual questionwithin the respective instrument, and whether the medical integrationsystem 102 determined the similar question in the respective instrumentto be an exact match or a similar match or some combination thereof.

In one exemplary embodiment, the print instrument section of the controlpanel allows users to select print parameters for the availableinstruments in the medical integration system 102. Before printing, theuser may select whether the instrument is printed in color, black &white or grey scale. The user also selects the font size of the contentwithin the instrument to suit the user's visual needs. Additionally, theprintable instrument defaults to the print in the user's defaultlanguage. Once the print parameters are selected, the user may downloadand print the instrument.

In another exemplary embodiment, a manage groups section of the controlpanel allows the user to manage the groups the user has invited toparticipate in a study, for example by editing (e.g., adding members toa group) and deleting groups. The manage groups sections may alsoinclude a groups list with the following fields: the group name, thedescription or purpose of the group, and the number of members in thegroup.

The control panel also includes, for example, a manage users section foradding and/or editing and/or deleting other users stored in the database126 of the medical integration system 102. Users may locate other usersby performing a first and/or last name user search and/or a clinicsearch (e.g., a search for the user based on the clinic with which theuser is associated). The medical integration system 102 determines thenames of users matching the entered search criteria and displays thesearch results based on first name, last name and/or clinic associationinformation.

In the exemplary embodiment described herein, the control panel includesan IRB management section which allows users to manage IRB submissionsby the physician generating the study. IRB management section eases therelationship and the transfer of documents between the IRB and thephysician. An instrument submission from a physician is added to the IRBmanagement section of the control panel, for example, along with studyinformation, consent forms and other required documentation required forIRB approval.

Access to the Medical Integration System

Users access the medical integration system 102 in various ways,including, for example, portal access, clinic access and email link oruniform resource location (“URL”) access. For portal access, users ofthe medical integration system 102 may be assigned account information(e.g., a username and a password) and may access the medical integrationsystem 102 through a web page or other access graphical user interfaceusing the assigned account information. To log into the user's account,the system 102 verifies the user's username and password combination.Once a valid combination has been given, the user may be prompted with asecurity question, which the user generated during account setup.

To keep up with the fast paced traffic that many clinics receive, themedical integration system 102 enables “clinic access,” in which aclinic authorizes system access on a computer (e.g., a dedicated machineor kiosk), thereby enabling a simplified login process. Once a computerhas been authorized and designated as a “clinic machine,” the loginprocess includes, for example, the user inputting the user's full nameand date of birth information into the system 102 to access the user'saccount or a particular instrument or to access the user's appointmentwith a physician.

Some patients prefer not to access the medical integration system 102through a username and password combination. To accommodate thesepatients, the medical integration system 102 enables email link and URLaccess by sending the user an electronic message (e.g., text message,email message or a SMS message) with an embedded link (e.g., URLhyperlink) through which the user accesses the system 102.

Clinics and other users (e.g., researchers and physicians) may alsopurchase a membership on a per clinic basis or on a per use basis toaccess the medical integration system 102. System membership allows auser to access the system 102 a specified number of times or anunlimited number of times.

In the exemplary embodiment described herein, users of the medicalintegration system 102 may be assigned a system role, which determinesthe permission level available to the user and the system functionalityprovided to the user. Some users are assigned multiple system roles anddifferent system roles for different clinics. The types of system rolesinclude, for example, clinic administrator, to manage the doctors andphysician assistants associated with a clinic and to perform otheradministrative tasks, including managing patients' addresses, paymentsand other system membership information. Other system roles include, forexample, the physician role, the physician assistant role, the patientrole and the IRB administrator role.

In one exemplary embodiment, system roles are associated with apermission level, including permission to generate a new study, join astudy, add patients to an existing study, invite doctors to join astudy, view the details of a study, post a message to all members withina study, assign instruments to users, generate a new patient profile fora clinic, view patients associated with a clinic, view clinic patientsfirst names and last names, join a group, generate a group, search forgroups, post messages to members in a group, edit the details of amembership, view memberships associated with a clinic, edit clinicmemberships, associate physicians, physician assistants, and clinicadministrators with a clinic, de-associate a user with a clinic and viewoverdue instruments.

In another exemplary embodiment, user actions performed within themedical integration system 102 are logged. Various user actionsperformed within the medical integration system 102 may be executed bythe system 102 through backend database queries. These queries may belogged and saved, providing a record of a user's identity and a timewhen information was accessed and providing a record of the type and thecontent of the accessed information.

Global Research Database

In the illustrated example of FIG. 1, as patients complete instruments,the medical integration system 102 receives instrument data (e.g.,answers to the questions included in the instruments) and stores thisreceived instrument data in one or more databases 126. In this example,patients access and complete instruments by accessing the instrumentsover the illustrated network 112 (e.g., the Internet or an intranet).Accordingly, the instrument data may be sent over the network 112 to themedical integration system 102 and may be stored in the database 126.Database 126 may include multiple databases running on the same machineor on different machines.

In the exemplary embodiment described herein, database 126 storesde-identified instrument data to protect patients' privacy and offer aresearch space for researchers worldwide. The instrument data may bede-identified by removing patient name, address, and other demographicdata to comply with the Health Insurance Portability and AccountabilityAct (“HIPAA”) and other privacy concerns. In doing so, the physicianlosses access to data associated with the physician's patients,instruments and submissions. To enable a physician who generated theinstrument to locate the physician's patients and access fields (e.g.,name and address information) that the physician is entitled to access,the de-identified data may be linked or otherwise associated with anidentifier, including without limitation a unique identifier, anumerical identifier and an encrypted identifier (e.g., an encryptedhidden code) (collectively referred to as an “encrypted identifier”herein, without limitation, for purposes of convenience.) Through theencrypted identifier, the physician may access, view and identifyinformation removed during the de-identification process and associatedwith the physician's patients. In one particular example, the encryptedidentifier may be generated by the system 102 using a random numbergenerator as is commonly known in the art.

In this exemplary embodiment, the encrypted identifier enablesphysicians to collaborate with each other and to plot de-identifiedpatients' data in comparison to other patients' data stored in thedatabase 126, including a physician's patients—which are identifiablethrough the encrypted identifier. In one particular example, a physicianmay plot how the physician's patients compare to patients in anotherworld/country/state/region/city.

In one exemplary embodiment, the database 126 may be structured toenable collaborators to generate a private research space within themedical integration system 102. Collaborators share data within theprivate research space and with others if they choose to do so.Additionally, the database 126 may include a public space. The publicspace includes, for example, the information and data which isaccessible to all users and members of the medical integration system102.

The medical integration system 102 also includes, for example, anintegration module, which is compliant with HIPAA and which allows thevarious processes, modules and sub-systems associated with the medicalintegration system 102 to communicate with one another and access thedata stored in the database 126.

Data Mining and Research Tools Module

In the illustrated example of FIG. 1, the medical integration system 102also includes a data mining and research tool module 120 (collectivelyreferred to as “research tool” herein, without limitation, for purposesof convenience), which may be an interactive component integrated withthe database 126 and may be used to retrieve data (e.g., informationregarding a particular study or instrument) stored in the database 126based on selected criteria. Through the research tool, a user comparesinstrument scores, views instrument scores, and views individual patientscores for a particular instrument for various timeline points in astudy.

The research tool includes, for example, a visual dynamic query tool tomine, retrieve and graph data from the various studies. The visualdynamic query tool includes a graphical user interface through whichusers of the medical integration system 102 generate queries and sendthe generated queries to the database 126. The research tool may alsoinclude a visual search tool though which a user of medical integrationsystem 102 enters search criteria (e.g., an age range for patients,gender information, race information or a study name). In thisparticular example, based on the entered search criteria, the medicalintegration system 102 may generate a search query and queries thedatabase 126 using the generated search query. The database 126 returnsto the system 102 data matching the criteria included in the searchquery. The returned data may be formatted to fit a desired visualcharting type. Additionally, the research tool 120 may executestatistical analysis rules to determine various statisticalcalculations, including, without limitation, average, mean, median andstandard deviation calculations, for study scores, instrument scores andscores linked to a patient. In this example, the statisticalcalculations include statistical data points, which may include averageor median calculations for a particular instrument within a study or fora particular subset of patients within a study.

FIG. 14 illustrates a particular exemplary embodiment described herein.Referring to FIG. 14, the medical integration system 102 generates agraphical user interface 370 that displays a real-time chartrepresentative of the executed queries and the data retrieved throughthe executed queries. The illustrated graphical user interface 370includes a section 372 in which the user selects the study for which areal-time chart is generated. The illustrated graphical user interface370 also includes a section 374 in which the user selects the physiciansfor which the user desires to view associated patient results. Theillustrated graphical user interface 370 includes a section 376 in whichthe user may select the instruments within a selected study for whichreal-time data is charted. Based on the user's selected criteria, theillustrated graphical user interface 370 displays a chart 378 of theinstrument scores associated with the selected instruments for theselected physicians.

In one exemplary embodiment, the medical integration system 102 includesan export data module to export and/or display data queried from thedatabase 126 in a pre-defined format (e.g., a format that publicationssupport and a format that conforms to medical journal and associationsguidelines). In another exemplary embodiment, the medical integrationsystem 102 also includes a multi-language module, which enablesdifferent language speaking researchers to communicate with each otherin the researchers' native language. Through the execution of languagerules which translate characters in a first language into a secondlanguage, the medical integration system 102 captures, executes,interprets and displays the review forums, instruments and queried datain the native language format of the researcher. Through themulti-language module, for example, the queried data is capable of beingdisplayed in another, different language.

In one particular example, a German researcher queries the medicalintegration system 102 in German and sends the queried data to anAmerican collaborator. The medical integration system 102 recognizes thelanguage difference between the sender (i.e., the German researcher) andthe recipient (i.e., the American collaborator) and displays and sendsthe query results in English to the American collaborator. Additionally,through the multi-language module, patients may complete instruments anddata is input and saved into the databases 126 in one language. Throughthe multi-language module, the completed instrument data may be viewedand/or queried by a user of the medical integration system 102 using asecond language.

Procedure Determination Module

The instruments described herein are often generated to test theefficacy of a procedure (e.g., a surgical procedure or a pharmaceuticaltreatment). In this case, through the use of the timelines describedabove, the system 102 may collect instrument data (through the variousquestions and answers) pertaining to a patient's symptoms, pain level,disability level or any other patient attributes (collectively referredto as “disability level” herein, without limitation, for purposes ofconvenience) before the procedure (“before procedure questions”) andafter the procedure (“after procedure questions”). In a preferredembodiment, a physician links a procedure identifier (e.g., a procedurecode or a procedure description) to the instrument. As the database 126collects and stores the patient's answers to the questions included inthe instrument, the answers are associated with the procedureidentifier. In this preferred embodiment, the database stores a list ofsymptoms associated with a procedure, where the symptoms are categorizedas those symptoms occurring before the procedure and those symptomsoccurring after the procedure.

In the exemplary embodiment described herein, the medical integrationsystem 102 may also include procedure assessment rules, which “grade” orclassify a procedure based on the determined efficacy of the procedure.In a particular example, the procedure assessment rules collate thepatient's symptoms before the procedure and the patient's symptoms afterthe procedure to determine if the patient's symptoms have improvedfollowing the procedure. For the procedures which improve a patient'ssymptoms, the procedure assessment rules may categorize those proceduresas being effective.

The medical integration system 102 also includes, for example, proceduredetermination rules to determine a recommended procedure or arecommended medical device based on patient's symptoms and aninstrument's score values, including, for example, symptom severityscore values and symptom improvement score values. The medicalintegration system 102 generates the procedure determination rules bydetermining the medical procedures that have been categorized aseffective (“effective medical procedures”) based on the score valuesassociated with a user's answers to various questions in an instrument.In one particular example, the system 102 determines the efficacy of ajoint replacement surgical procedure. A before procedure questionincludes the following question: “What level of pain are youexperiencing in your joint?” The symptom improvement score valuesassociated with this question include, for example, “1”, “2” and “3”. Ascore value of “1” is indicative of a high pain level. A score value of“2” is indicative of a moderate pain level. A score value of “3” isindicative of a low pain level. In this example, the after procedurequestion includes the same question (i.e., “what level of pain are youexperiencing in your joint?”) as the before procedure question with thesame associated score values. If a patient answered the before procedurequestion with a “high pain level” answer (i.e., a symptom improvementscore value of “1”) and the after procedure question with the “low painlevel” answer (i.e., a symptom improvement score value of “3”), thesystem 102 categorizes the joint replacement surgical procedure as aneffective medical procedure, based on the symptom improvement scorevalue increasing from a value of “1” to a value of “3.”

In the exemplary embodiment described herein, for the effective medicalprocedures, the medical integration system 102 generates a proceduredetermination rule including the symptoms patients experienced beforethe procedure. In a particular example, a procedure determination rulefor joint replacement includes the following instructions “Ifsymptoms==joint pain or joint difficulty or joint swelling, thenprocedure==joint replacement.” When a user inputs a patient's symptomsinto the system 102, the procedure determination module 122 in thisexample executes the procedure determination rules by matching one ormore characters included in the input symptoms to the symptoms includedin the procedure determination rules. Based on returned symptom matches,the procedure determination rules generate a list of recommendedprocedures.

Patient Flow Module

Busy clinics experience patient overload that results in patientswaiting longer to see a physician. As wait times increase, patientsatisfaction with a patient's physician and/or clinic experiencedecreases. In the illustrated example of FIG. 1, the patient flow module124 improves patient satisfaction by efficiently tracking actions apatient needs to take and instruments a patient needs to fill out priorto engaging with a physician. The illustrated patient flow module 124 isintegrated with various aspects of the medical integration system 102,including, for example, the database 126. Through querying the database126, the patient flow module 124 determines the instruments a patientneeds to complete prior to a consultation with a physician. The patientflow module 124 may also track a patient's steps and activities from thetime the patient arrives at the clinic, registers with the clinic, waitsto see a nurse, is seen by a nurse and is moved to an examination room.In a preferred embodiment, the module 124 provides clinics (e.g.,physicians and nurses and health care providers) feedback and guidanceas to the next action to be performed and the information the patientneeds to provide the clinic, physician or nurse. By tracking thisinformation, the patient flow module 126 facilitates moving patientsthrough the clinic check-in process more quickly and thereby alleviatespatient wait time. The module 124 may also manage patient load byalerting health care providers of the steps that need to be taken tomove patients through an examination.

To facilitate quick and easy check-in once a patient has arrived at aclinic, the medical integration system 102 may generate a portalenvironment through which a patient accesses the medical integrationsystem 102 to view the patient's appointment by entering patientinformation, including, for example, name information, date of birthinformation and/or selecting the name of the patient's doctor at theclinic. In a preferred embodiment, using the entered information, themedical integration system 102 queries the database 126 for thepatient's appointment and determines the physician with which thepatient is scheduled for an appointment. The module 126 may also querythe database 126 to determine a listing of one or more instruments whichthe patient's physician requires the patient to complete prior to theconsultation.

FIG. 15 illustrates a particular exemplary embodiment described herein.Referring to FIG. 15, the patient flow module 124 performs 380 variousactions in tracking a user's appointment with a physician. The patientflow module 124 queries 381 the database 126 for a list of instrumentslinked to a physician thereby determining the instruments a physicianrequires a patient to complete prior to an examination. In thisillustrated example, based on the query results, the patient flow module124 determines 382 a list of instruments the patient needs to completeand the content of answers for the questions in a completed instrument.Additionally, in this illustrated example, the patient flow module 124generates 383 a list of instruments the patient needs to completes andgenerates a graphical user interface that includes this list, asdescribed in further detail below.

In the exemplary embodiment described herein, the database 126 may storemedical assessment rules (i.e., instructions for how the system 102, thepatient flow module 124 or a process executed by the system 102 analyzesand assesses the content of a user's answers to questions included in aninstrument.) Through execution of the medical assessment rules, thesystem 102 determines, without limitation, diagnosis information,treatment information and medication information for a patient. Thesystem 102 generates the medical assessment rules by receiving a listingof symptoms (e.g., symptoms regarding pain) and a listing of medicaldiagnoses associated with the symptoms. In a particular example of amedical assessment rule, if the system receives patient symptomsindicating that the patient is experiencing pain when opening a jar,then the patient is assigned an assessment of “wrist pain.”

In the illustrated example of FIG. 15, the patient flow module 124retrieves 138 the medical assessment rules from the database 126 andapplies 385 the assessments rules to the content and characters includedin the patient's answers. Through the application of the medicalassessment rules, the patient flow module 124 may generate a list ofdiagnoses and treatments applicable to the patient.

FIG. 16 illustrates a particular exemplary embodiment described herein.Referring to FIG. 16, the medical integration system 102 generates agraphical user interface 387 that when rendered on a display devicedisplays a visual representation 388 of the names of the forms that needto be filled out by the patient. The illustrated graphical userinterface 387 also includes links 389 a, 389 b, selection of which causethe medical integration system 102 to generate another graphical userinterface that displays the contents of the instrument associated withthe selected link. In a preferred embodiment, as a patient fills out andcompletes the selected instruments, the patient's entered data andanswers are saved in the database 126, allowing the patient's entereddata and answers to be viewed both by the patient's physician and byother researchers and collaborators querying the database 126, asdescribed above.

Referring to the illustrated example of FIG. 1, the patient flow module124 also notifies health care providers, including physicians andnurses, of the instruments a patient has already completed. In apreferred embodiment, the illustrated patient flow module 124 generatesan electronic message including a list of the names of the completedinstruments and the system 102 sends the electronic message to thehealth care provider. Additionally, the module 124 enables the healthcare providers to add additional instruments to the list of instrumentsfor a patient to complete. Referring to the illustrated example of FIG.16, these newly added instruments may be displayed in graphical userinterface 387, notifying the patient of the newly added instruments.

Referring back to the illustrated example of FIG. 1, the patient flowmodule 124 facilitates information flow and continuing of medical careby enabling health care providers to send completed instruments to otherhealth care providers and external and internal EMR systems. FIG. 17illustrates a particular exemplary embodiment described herein.Referring to FIG. 17, the medical integration system 102 generates agraphical user interface 390, with illustrated section 392 displaying alist of the instruments with which a patient is currently associated andwhich the patient needs to complete. Through selection of theillustrated link 394, a health care provider adds another instrument tothe list of instruments the patient needs to complete. Section 396 ofthe illustrated graphical user interface 390 enables a user to send acompleted instrument to a recipient (e.g., a physician), throughselection of link 398, thereby facilitating continuity of medical care.

In the illustrated example of FIG. 1, the patient flow module 124interacts with appointment systems and EMR systems 128 to enable healthcare providers to manage appointments, monitor patient flow in a clinicand perform the proper processes and tasks needed move patients throughthe clinic. In a particular example, the system 102 retrieves aphysician's appointment schedule from the database 126 or anotherappointment schedule system. A physician's appointment schedule includesthe names of the patients schedule for a visit with the physicians. Inthis example, if a physician is overbooked or double booked, the module124 enables a user to associate one or more of the physician'sappointments with another physician, thereby decreasing the amount bywhich one physician is overbooked and decreasing the amount of time apatient needs to wait to see a physician.

In the exemplary embodiment described herein, the patient flow module124 may query the database 126 for a list of instruments completed by aparticular patient. The module 124 may apply the medical assessmentrules, described above, to analyze the patient data (e.g., answers tothe questions included in the instrument) and to generate a visualrepresentation of the health problems and issues experienced by thepatient. In this embodiment, the visual representation enables healthcare providers to quickly and accurately assess a patient's condition,saving the health care provider time in diagnosing a patient's healthproblems.

FIG. 18 illustrates a particular exemplary embodiment described herein.Referring to FIG. 18, the medical integration system 102 generates agraphical user interface 400 including one or more visualrepresentations 402 of a section of the patient's anatomy in which thepatient is experiencing pain or a medical issue. Section 404 of theillustrated graphical user interface 400 chronicles these visualrepresentations 402 as a function of time or medical visits with thephysician or clinic. Illustrated section 406 of the graphical userinterface 400 includes, for example, metrics associated with a patient'streatment, diagnosis and symptoms. Illustrated section 406 is alsorepresented as a function of time or visits in comparison to section404.

Embodiments can be implemented in digital electronic circuitry, or incomputer hardware, firmware, software, or in combinations thereof.Apparatus of the invention can be implemented in a computer programproduct tangibly embodied or stored in a machine-readable storage devicefor execution by a programmable processor; and method actions can beperformed by a programmable processor executing a program ofinstructions to perform functions of the invention by operating on inputdata and generating output. The invention can be implementedadvantageously in one or more computer programs that are executable on aprogrammable system including at least one programmable processorcoupled to receive data and instructions from, and to transmit data andinstructions to, a data storage system, at least one input device, andat least one output device. Each computer program can be implemented ina high-level procedural or object oriented programming language, or inassembly or machine language if desired; and in any case, the languagecan be a compiled or interpreted language.

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read-only memory ora random-access memory or both. The essential elements of a computer area processor for executing instructions and one or more memory devicesfor storing instructions and data. Generally, a computer will alsoinclude, or be operatively coupled to receive data from or transfer datato, or both, one or more mass storage devices for storing data, e.g.,magnetic, magneto optical disks, or optical disks. Computer readablemedia for embodying computer program instructions and data include allforms of non-volatile memory, including by way of example semiconductormemory devices, e.g., EPROM, EEPROM, and flash memory devices; magneticdisks, e.g., internal hard disks or removable disks; magneto opticaldisks; and CD ROM and DVD-ROM disks. The processor and the memory can besupplemented by, or incorporated in special purpose logic circuitry. Anyof the foregoing can be supplemented by, or incorporated in, ASICs(application-specific integrated circuits).

To provide for interaction with a user, embodiments can be implementedon a computer having a display device, e.g., a LCD (liquid crystaldisplay) monitor, for displaying information to the user and a keyboardand a pointing device, e.g., a mouse or a trackball, by which the usercan provide input to the computer. Other kinds of devices can be used toprovide for interaction with a user as well; for example, feedbackprovided to the user can be any form of sensory feedback, e.g., visualfeedback, auditory feedback, or tactile feedback; and input from theuser can be received in any form, including acoustic, speech, or tactileinput.

Embodiments can be implemented in a computing system that includes aback end component, e.g., as a data server, or that includes amiddleware component, e.g., an application server, or that includes afront end component, e.g., a client computer having a graphical userinterface or a Web browser through which a user can interact with animplementation of embodiments, or any combination of such back end,middleware, or front end components. The components of the system can beinterconnected by any form or medium of digital data communication,e.g., a communication network. Examples of communication networksinclude a local area network (LAN) and a wide area network (WAN), e.g.,the Internet.

The system and method use the “World Wide Web” (Web or WWW), which isthat collection of servers on the Internet that utilize the HypertextTransfer Protocol (HTTP). HTTP is a known application protocol thatprovides users access to resources, which may be information indifferent formats such as text, graphics, images, sound, video,Hypertext Markup Language (HTML), as well as programs. Uponspecification of a link by the user, the client computer makes a TCP/IPrequest to a Web server and receives information, which may be anotherWeb page that is formatted according to HTML. Users can also accessother pages on the same or other servers by following instructions onthe screen, entering certain data, or clicking on selected icons. Itshould also be noted that any type of selection device known to thoseskilled in the art, such as check boxes, drop-down boxes, and the like,may be used for embodiments using web pages to allow a user to selectoptions for a given component. Servers run on a variety of platforms,including UNIX machines, although other platforms, such as Windows2000/2003, Windows NT, Sun, Linux, and Macintosh may also be used.Computer users can view information available on servers or networks onthe Web through the use of browsing software, such as Firefox, NetscapeNavigator, Microsoft Internet Explorer, or Mosaic browsers. Thecomputing system can include clients and servers. A client and serverare generally remote from each other and typically interact through acommunication network. The relationship of client and server arises byvirtue of computer programs running on the respective computers andhaving a client-server relationship to each other.

Other embodiments are within the scope and spirit of the descriptionclaims. In one embodiment, the rules described herein (e.g., theprocedure determination rules or the medical assessment rules) areexecuted by a rules engine included in the system 102. In anotherembodiment, data collected by the system 102 through the instruments isstored in an EMR system 128. The research tool may then query the EMRsystem 128 for patient data matching one or more patient criteria.Through the network 112, the matching data is returned to the system 102and the research tool processes and analyzes the returned data. In yetanother embodiment, the techniques described herein are used togenerate, review and validate instruments pertaining to various fields(e.g., the veterinary field, the legal field and the financial servicesfield) and collect and retrieve data for the instruments pertaining tothe various fields. In still another embodiment, the instrumentgeneration module 116, the instrument validation module 118, theresearch tools module 120, the procedure determination module 122 andthe patient flow module 124 are integrated together through variouscommunication channels and/or are implemented as an instrumentgeneration system, an instrument validation system, a research toolssystem, a procedure determination system and a patient flow system(collectively referred to as “the systems” herein, without limitation,for the purposes of convenience), with each system including one or moreservers or computing devices and the systems being integrated togetherthrough various communication channels and/or network connections.

Additionally, due to the nature of software, functions described abovecan be implemented using software, hardware, firmware, hardwiring, orcombinations of any of these. Features implementing functions may alsobe physically located at various positions, including being distributedsuch that portions of functions are implemented at different physicallocations. The use of the term “a” herein and throughout the applicationis not used in a limiting manner and therefore is not meant to exclude amultiple meaning or a “one or more” meaning for the term “a.”Additionally, to the extent priority is claimed to a provisional patentapplication, it should be understood that the provisional patentapplication is not limiting but includes examples of how the techniquesdescribed herein may be implemented.

What is claimed is:
 1. A method comprising: transmitting, to a firstdistributed device, information indicative of a medical study instrumentthat includes one or more questions that are each associated with acorresponding linked score value, wherein assigning a linked score to ananswer to a question creates a logical relationship between the linkedscore and the answer to the question; receiving, from the firstdistributed device, first information indicative of answers to each ofthe one or more questions; obtaining, from a second distributed device,second information indicative of a score rule that defines howparticular answers to the one or more questions are valued based on atleast one of the linked score values; correlating, by one or moreprocessing devices, the first information indicative of answers receivedfrom the first distributed device with the second information indicativeof the scoring rule obtained from the second distributed device todetermine a score value for each of the one or more questions,respectively, based on at least one of the linked score values thatcorresponds to the one or more questions; determining, by the one ormore processing devices, a score for a first answer to a first questionsatisfies a predetermined threshold, wherein the score for the firstanswer is determined based on the first information indicative ofanswers received from the first distributed device and the secondinformation indicative of the scoring rule obtained from the seconddistributed device; in response to the determination that the score forthe first answer of the first question has satisfied the predeterminedthreshold: transmitting, by the one or more processing devices, anotification in real-time to a user device of a user that is configuredto, when processed by the user device, cause the user device to modify adisplay of at least one graphical element on a user interface of theuser device to alert the user of the user device that information,representing the first answer to the first question of the medical studyinstrument, has been received that satisfies the predeterminedthreshold; and generating, by the one or more processing devices,another score for at least one other question of at least one othermedical study instrument based on the first information.
 2. The methodof claim 1, the method further comprising: determining, by the one ormore processing devices, a score for the medical study instrument,wherein determining the score for the medical study instrumentcomprises: determining one or more weighted score values.
 3. The methodof claim 1, the method further comprising: determining, by the one ormore processing devices, a score for the medical study instrument,wherein determining the score for the medical study instrumentcomprises: determining a weighted score for the medical studyinstrument.
 4. The method of claim 1, further comprising: generating anotification message indicating that one or more of the questionsincluded in the medical study instrument need to be answered; andsending the notification message to one or more devices used by one ormore participants.
 5. The method of claim 1 further comprising:generating by the one or more processing devices information for amultimedia module to explain one or more features of a particularquestion.
 6. The method of claim 1, further comprising: generating bythe one or more processing devices information for a graphical userinterface that when rendered on a display device comprises a visualrepresentation of the medical study instrument.
 7. One or moremachine-readable hardware storage devices storing instructions that areexecutable by one or more processing devices to perform operationscomprising: transmitting, to a first distributed device, informationindicative of a medical study instrument that includes one or morequestions that are each associated with a corresponding linked scorevalue, wherein assigning a linked score to an answer to a questioncreates a logical relationship between the linked score and the answerto the question; receiving, from the first distributed device, firstinformation indicative of answers to each of the one or more questions;obtaining, from a second distributed device, second informationindicative of a score rule that defines how particular answers to theone or more questions are valued based on at least one of the linkedscore values; correlating, by one or more processing devices, the firstinformation indicative of answers received from the first distributeddevice with the second information indicative of the scoring ruleobtained from the second distributed device to determine a score valuefor each of the one or more questions, respectively, based on at leastone of the linked score values that corresponds to the one or morequestions; determining, by the one or more processing devices, a scorefor a first answer to a first question satisfies a predeterminedthreshold, wherein the score for the first answer is determined based onthe first information indicative of answers received from the firstdistributed device and the second information indicative of the scoringrule obtained from the second distributed device; in response to thedetermination that the score for the first answer of the first questionhas satisfied the predetermined threshold: transmitting, by the one ormore processing devices, a notification in real-time to a user device ofa user that is configured to, when processed by the user device, causethe user device to modify a display of at least one graphical element ona user interface of the user device to alert the user of the user devicethat information, representing the first answer to the first question ofthe medical study instrument, has been received that satisfies thepredetermined threshold; and generating, by the one or more processingdevices, another score for at least one other question of at least oneother medical study instrument based on the first information.
 8. Theone or more machine-readable hardware storage devices of claim 7,wherein the operations further comprise: determining, by the one or moreprocessing devices based on the correlated information, a score for themedical study instrument, wherein determining score values comprises:determining one or more weighted score values.
 9. The one or moremachine-readable hardware storage devices of claim 7, wherein theoperations further comprise: determining, by the one or more processingdevices based on the correlated information, a score for the medicalstudy instrument, wherein determining the score for the medical studyinstrument comprises: determining a weighted score for the medical studyinstrument.
 10. The one or more machine-readable hardware storagedevices of claim 7, wherein the operations further comprise: generatinga notification message indicating that one or more of the questionsincluded in the medical study instrument need to be answered; andsending the notification message to one or more devices used by one ormore participants.
 11. The one or more machine-readable hardware storagedevices of claim 7, wherein the operations further comprise: generatingby the one or more processing devices information for a graphical userinterface that when rendered on a display device comprises a visualrepresentation of the medical study instrument.
 12. An electronic systemcomprising: one or more processing devices; and one or moremachine-readable storage devices storing instructions that areexecutable by one or more processing devises to perform operationscomprising: transmitting, to a first distributed device, informationindicative of a medical study instrument that includes one or morequestions that are each associated with a corresponding linked scorevalue, wherein assigning a linked score to an answer to a questioncreates a logical relationship between the linked score and the answerto the question; receiving, from the first distributed device, firstinformation indicative of answers to each of the one or more questions;obtaining, from a second distributed device, second informationindicative of a score rule that defines how particular answers to theone or more questions are valued based on at least one of the linkedscore values; correlating, by one or more processing devices, the firstinformation indicative of answers received from the first distributeddevice with the second information indicative of the scoring ruleobtained from the second distributed device to determine a score valuefor each of the one or more questions, respectively, based on at leastone of the linked score values that corresponds to the one or morequestions; determining, by the one or more processing devices, a scorefor a first answer to a first question satisfies a predeterminedthreshold, wherein the score for the first answer is determined based onthe first information indicative of answers received from the firstdistributed device and the second information indicative of the scoringrule obtained from the second distributed device; in response to thedetermination that the score for the first answer of the first questionhas satisfied the predetermined threshold: transmitting, by the one ormore processing devices, a notification in real-time to a user device ofa user that is configured to, when processed by the user device, causethe user device to modify a display of at least one graphical element ona user interface of the user device to alert the user of the user devicethat information, representing the first answer to the first question ofthe medical study instrument, has been received that satisfies thepredetermined threshold; and generating, by the one or more processingdevices, another score for at least one other question of at least oneother medical study instrument based on the first information.
 13. Theelectronic system of claim 12, wherein the operations further comprise:determining, by the one or more processing devices based on thecorrelated information, a score for the medical study instrument,wherein determining score values comprises: determining one or moreweighted score values.
 14. The electronic system of claim 12, whereinthe operations further comprise: determining, by the one or moreprocessing devices based on the correlated information, a score for themedical study instrument, wherein determining the score for the medicalstudy instrument comprises: determining a weighted score for the medicalstudy instrument.
 15. The electronic system of claim 12, wherein theoperations further comprise: generating a notification messageindicating that one or more of the questions included in the medicalstudy instrument need to be answered; and sending the notificationmessage to one or more devices used by one or more participants.
 16. Theelectronic system of claim 12, wherein the operations further comprise:generating by the one or more processing devices information for agraphical user interface that when rendered on a display devicecomprises a visual representation of the medical study instrument. 17.The method of claim 1, wherein the one or more questions of the medicalstudy instrument include at least one condensed question that representsmultiple similar questions across multiple medical study instruments.18. The one or more machine-readable hardware storage devices of claim7, wherein the one or more questions of the medical study instrumentinclude at least one condensed question that represents multiple similarquestions across multiple medical study instruments.
 19. The electronicsystem of claim 12, wherein the one or more questions of the medicalstudy instrument include at least one condensed question that representsmultiple similar questions across multiple medical study instruments.